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ABSTRACT 

This paper discusses the comprehensive evaluation of 
a new Ed.D, Program in Educational Leadership housed in East Carolina 
University's (ECU) School of Education. The program proposed to 
prepare students for professional life through development of career 
leadership skills; utilization, development, and dissemination of 
research; and service to the community through internship and 
research programs. The evaluation of the program was conducted 
through a survey and telephone interviews of a sample of doctoral 
students (N=38) enrolled in Spring 1996, graduates of the program 
(N=8) and the advisory committee members (N=l2) for the program. The 
respondents in this study indicated that the academic rigor, the 
advising, the interactions among the students and faculty, and the 
schedule and structure of the program were effective. They felt, 
however, that the research and methodology component and the 
dissertation process could be strengthened and that possible gender 
and racial insensitivity may impact the effectiveness of the program. 
The data suggest that an adjustment to changing interpersonal 
dynamics between professors and students and deliberate planning are 
essential to the effective implementation of this type of program. 

The faculty has already begun to make changes in response to this 
study, including reconceptualization of the research sequence and 
development of new procedures for the approval of dissertation 
proposals. (Contains 13 references.) (PRW) 
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This paper discusses the comprehensive evaluation of a relatively new Ed. D. 
Program in Educational Leadership that is housed in East Carolina University’s (ECU) 
School of Education (SOE). The proposal for the program stated. 

It will prepare students for full professional lives through its emphasis upon the 
development of career leadership skills. It will focus on the utilization, 
development, and dissemination of research. Through its internship and research 
programs, it will provide services to the community. (East Carolina University, 
1990, pp. 8-9) 

The SOE faculty wanted to produce educational leaders who could develop and lead self- 
renewing organizations. 

Self-renewing Organizations 

Human beings generally have a strong commitment to the status quo. Schon 
(1971) calls it a desire for the stable state. 

Belief in the stable state is belief in the unchangeability, the constancy of central 
aspects of our lives, or belief that we can attain such a constancy. Belief in the 
stable state is strong and deep in us. We institutionalize it in every social domain. 
We do this in spite of our talk about change and our approval of dynamism. 
(Schon, 1971, p. 1) 

Most organizations develop a culture that tries to protect the stable state in 
organizational members, i.e., protect them from information overload and uncertainty 
(Schein, 1985; Schon, 1971). Yet the rapid rate of change in most societies suggests that 
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the reality of the stable state will be lost to most people (Schon, 1971). If an 
organization’s culture, structure, and procedures do not adapt to meet changing needs, 
the organization will become ineffective (Schein, 1985) 

Argyris (1983) argued that an organization could meet the needs of its members 
and effectively meet its external challenges by developing the ability to learn as an 
organization, i.e., renew itself. Argyris and Schon (1974) called this ability double-loop 
learning. Double-loop learning attempts to create situations “in which the basic 
assumptions behind ideas or policies are confronted, in which hypotheses are tested 
publicly, and in which the processes are disconfirmable, not self-sealing” (Argyris, 1983, 
pp. 103-104). 

The end result should be increases in the effectiveness of decision making, in the 
monitoring of decisions and policies, and in the probability that errors and failures 
will be communicated openly and that actors will learn from the feedback. 
(Argyris, 1983, p. 104) 

A necessary condition for an organization that wants to pursue double-loop 
learning is having organizational members who possess that knowledge and skill. In 
other words, if we want the schools of the 21st century to be self-renewing organizations 
we must produce educational leaders whose actions are a function of these governing 
variables: (a) valid information, (b) free and informed choice, (c) internal commitment to 
the choice, and (d) the constant monitoring of implementation (Argyris, 1983). The ECU 
Ed. D. Program in Educational Leadership attempted to prepare educational leaders who 
met those necessary conditions. 
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ECU’s Ed. D. Program in Educational Leadership 

The Board of Governors of the University of North Carolina gave East Carolina 
University permission to establish a Doctor of Education degree program in 1990. The 
Program leads to an Ed. D in Educational Leadership. It is designed for senior school 
administrators. The Program is based in the School of Education and administered by the 
Department of Educational Leadership (LEED), but it uses courses and faculty from other 
schools within the University. The program has five components, which combine for a 
minimum of 60 hours beyond the Masters Degree: (a) The leadership core requires a 
minimum of 24 credit hours, (b) The cognate area requires 12 credit hours. This area is 
designed to complement the student’s major area of study and his career goals, (c) The 
one-year supervised internship is designed to provide leadership experiences in 
educational settings. The internship experience relies heavily on reflective journals and 
action research. The internship generates 6 credit hours, (d) The research core is 
designed to prepare students to evaluate, synthesize, and apply research to research 
problems and to generate new research through carefully designed studies. This 
component has a minimum of 12 credit hours, (e) The doctoral dissertation is designed to 
address an area of inquiry that serves not only to advance the student’s knowledge but 
also benefits educational organizations. The dissertation generates 6 credit hours. 

The faculty in Educational Leadership worked in collaboration with faculty in the 
School of Education and other units within the University to develop a program with high 
expectations and high support. The faculty wanted to emphasize the translation of theory 
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into practice (Argyris & Schon, 1974) that would benefit education in North Carolina. 
They also wanted to persuade students to believe in the efficacy of disciplined inquiry 
(Jaeger, 1988) through collaborative efforts and through modeling (Sergiovanni, 1987). 
The LEED faculty emphasized the use of problem-based learning in the Ed. D. 
curriculum (Barrows, 1985; Pajak, Tanner, Rees, & Holmes., 1995; Wilkerson & 
Felletti, 1989). The following factors guide the instructional process: 

1 . Students encounter or identify an issue which offers a problem of 
professional practice. 

2. They engage in problem-solving in a group interactive process. 

3. They apply their new information and knowledge to the problem. 

4. They summarize what has been learned (Pajak, et al., 1995). 

In other words, the Educational Leadership faculty try to prepare students to become 
double-loop learners and to model double-loop learning for the students (Argyris & 
Schon, 1974). 

Educational Leadership Department fLEEDl 

The LEED Department offers four graduate degree programs, MAED in 
Supervision, MA— Masters of School Administration, Ed. S. in Educational 
Administration and Supervision, and Ed. D in Educational Leadership. Ten faculty 
positions have resided in the Department. 

Program Implementation 

The first cohort of 1 1 doctoral students enrolled in Fall 1990. Sixty-five students 
have enrolled in the program in six cohorts, i.e., 1990, 1991, 1992, 1993, 1994, 1995. 
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Fourteen students have graduated from the program (see Appendix A), 1 died, 5 have 
withdrawn— 8 % attrition, and 7 did not take courses in Spring 1996. Thirty-eight 
students were enrolled in courses during the Spring of 1996. 

The Evaluation Process 

A team of three faculty members and one graduate student in the School of 
Education, East Carolina University doctoral program conducted the evaluation during 
1996. Given the background and goals of the Ed. D. Program in Educational Leadership, 
the evaluation team developed several evaluation questions: 

1 . Was the academic program rigorous? 

2 Was the advising in the program effective? 

3 What was the climate and culture of the program? 

4. What was the nature of the interaction among the faculty and students? 

5 Was the design and scheduling of the program effective? 

6. Did the students experience personal growth? 

7. Did the students experience professional growth? 

8. What suggestions did the students have to improve the program? 

The evaluation team addressed these questions by surveying a census sample of the 
doctoral students who were enrolled during Spring 1996, conducting telephone interviews 
with a sample of students, conducting telephone interviews with graduates of the 
program, and conducting interviews with the advisory committee for the program. 
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Survey Results 

The 38 students enrolled in Spring 1996 were the population for a student survey 
(see Appendix B). The survey addressed 10 issues: (a) academic rigor, (b) advising, (c) 
cohesion of cohort group, (d) respect and courtesy of the faculty, (e) access to LEED 
faculty, (f) access to other School of Education faculty, (g) translation of theory to 
practice, (h) scheduling, (i) research methodology, and (j) recommendations. 

The results of the student survey were positive. The student survey had nine 
Likert scale positive statements about the program. The respondents were asked to rate 
the positive statements from strongly agree (1) to strongly disagree (4). The 30 
respondents, 79% of the population, agreed or strongly agreed with the nine Likert scale 
questions (see Figure 1) 




□ Rigor 

■ Advising 

□ Cohesion 

□ Respect 

■ Access 1 

□ Access 2 

■ Theory 

□ Schedule 

□ Research 




Figure 1 . Histogram of Scores on Likert Scale Questions of Student Survey 
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Twenty-six of the respondents made comments or suggestions. Two themes were 
mentioned by several students: (a) The research sequence should place more emphasis on 
the practical application of research methodology, (b). Students should receive better 
guidance and support through the dissertation process, (c) The admissions criteria should 
be broadened to include classroom teachers, and (d) general positive comments about he 
program. Other comments addressed the following issues: 

1 . More opportunities for cohort cohesion 

2. Reduce the size of the required seminars 

3. Use more diverse instructional strategies 

4. Offer more courses more often. 

5. Address gender and racial bias. 

Telephone Interviews of Current Students 

The sample for the telephone interviews was a random sample of 20% of the 38 
students who were enrolled in Spring 1996. The interview protocol (see Appendix C) 
addressed the eight evaluation questions. 

All the students in the sample (n=8) took part in the telephone interviews. 

Five interview questions addressed academic rigor. The responses were positive. 
However, two responses expressed concern about the quality of the methodology 
sequence in the program. Four interview questions addressed the quality and content of 
advising. The responses were very positive. Three questions addressed the cohesion and 
cooperation of the cohorts. The responses were mixed. The majority of responses were 
positive, but several responses indicated that they felt that their cohort lacked cohesion. 
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Five questions addressed the interaction with faculty. The responses were positive, but 
one respondent indicated some problems with one faculty member. Five questions 
addressed the internship. The responses were positive. Four questions addressed 
personal growth and development and three questions addressed professional growth and 
development. All the responses in both categories were positive. Three questions 
addressed the residence requirement. The responses were very mixed.. The responses 
ranged from “Likes it”; “no problem with it” to “Do away with them; formality— no 
useful purpose”. Other responses indicated that the residence requirement had some 
positive benefit on professional development, but lack of adequate funding for the 
residence requirement had negative consequences for respondents and their families. 

Two questions addressed the scheduling of courses. The responses were positive, but 
some respondents indicated that they would like smaller course sections and more 
publicity for the multi-year course schedule. Two questions addressed socialization in 
the culture of the program. The responses were positive. Two questions addressed 
support and assistance. The responses to these questions were positive. The last section 
of the interview addressed recommendations. The recommendations addressed adding 
more technology to the curriculum, improving the integration of methodology with other 
courses, and integration of some additional topics and procedures to the curriculum. 

The responses to the interview protocol were generally very positive. The 



recommendations seemed rational and reasonable. 
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Telephone Interviews of Graduates. 

The interview protocol for the graduates of the ECU Ed. D. Program in 
Educational Leadership has three questions: (a). Please describe your experience in the 
Ed. D. Program at ECU? (b). What impact did the Program have on your personally and 
professionally? (c). How can the Program be improved? Eight of the 14 students who 
had completed the program by Spring 1996 took part in the telephone interviews. The 
respondents were extremely positive about their experiences in particular and the Ed. D. 
Program in general. They believed that the ECU Ed. D. Program in Educational 
Leadership was equal to or superior to similar programs in North Carolina. They 
believed that their programs of study had prepared them well for their current positions. 
They recommended that new students in the Ed. D. Program should start planning early 
for their dissertation. Also, they suggested that the new students should receive an 
orientation from graduates and more experienced students. In addition, the respondents 
emphasized the use of possible dissertation topics in research classes would make their 
exercises more practical. In general, the respondents were very positive about their 
experiences in the ECU Ed. D. Program in Educational Leadership. 

Telephone Interviews of Advisory Committee 

The interview protocol for the Advisory Committee had two questions: (a). What 
is your impression of ECU’s Ed.D. Program in Educational Leadership? (b). Is the 
Program operating the way it was intended? The original Advisory committee for the 
ECU Ed. D. Program in Educational Leadership had 25 members. Several members had 
retired and moved from the area, one member is deceased, and several did not return 
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telephone calls. Twelve members of the Advisory Committee responded to the two 
questions. The respondents believed that the program is doing well, but that it needs to 
change to keep up with technological growth and expectations. They believed program 
implementation is close to its original design with one big exception. The State of North 
Carolina never provided the funds to support full-time doctoral study that were promised 
up to the week the first doctoral student entered the program. In general, the Advisory 
Committee had very positive perceptions of the ECU’s Ed. D. Program and the Program 
factors that ECU could control. 

Summary of Evaluation Findings 

The data from the survey and telephone interviews indicate that respondents 
believed that the academic rigor, the advising, and the climate and culture of the ECU Ed. 
D. Program in Educational Leadership were good and effective. Respondents did 
mention a concern about the integration of appropriate research methodology into overall 
program of study. 

The perceptions of the interactions among faculty and students were positive, but 
student cohorts could be more cohesive and students could feel more comfortable about 
communicating with faculty. Perceptions of the structure and schedule of the program 
were positive, but the unfunded residence requirement was problematic. There was 
agreement that the residence requirement was beneficial to professional development, but 
there was also agreement that the unfunded residence requirement added significant 
stress to program participants and their families. 
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All the students and graduates of the Ed. D. Program reported personal and 
professional growth and development. The recommendations from the respondents 
suggested that the ECU faculty need to continue their emphasis on data-based decision 
making, program planning, and problem-based learning and to strengthen their 
communication and interpersonal skills. 

Reflections 

The 14 students who had completed the Ed. D. Program in Educational 
Leadership at the time of this analysis developed dissertations that investigated questions 
that intrigued them and addressed the needs of public education (see Appendix A). The 
LEED faculty’s efforts to incorporate problem -based learning and technology into the Ed. 
D. curriculum conform with best practices in the field (Geltner, 1995; Pajak et al., 1995; 
Wendel, 1992). In addition, the use of cohorts and field-based internships are consistent 
with emerging trends in the field (Hackman & Price, 1995). 

The student survey and interviews indicated that students believe that the program 
is rigorous, the advising is effective, the interactions among the students and faculty are 
effective, and the schedule and structure of the program are effective. The comments 
suggested that the faculty could strengthen the research and methodology component and 
the dissertation process. In addition, some elements of the culture, such as perceived 
gender and racial insensitivity, may impact the effectiveness of ECU’s Ed. D. Program in 
Educational Leadership. These findings are consistent with conclusions of an assessment 
of a relatively new Ed. D. program in Educational Leadership at the University of 
Georgia. 
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A major conclusion drawn from the data is that an adjustment to changing 
interpersonal dynamics between professors and students and deliberate planning 
are essential to the effective implementation of this type of program. (Pajak, et ah, 
1995, p. 18) 

The observations of the evaluation team suggest that the LEED faculty could 
improve how we model double-loop learning, i.e., we do not always practice what we 
preach. For example, we need to improve collaboration with all School of Education 
faculty and our graduate students. We need to work harder at following the guiding 
principles of double-loop learning: (a) valid information for decisions, (b) free and 
informed choice, (c) internal commitment to choice, and (d) the constant monitoring of 
implementation (Argyris, 1983). 

Implications 

The faculty in the School of Education has already begun to reconceptualize the 
research sequence and its relationship to the other elements of the Ed. D. Program of 
study. In addition, the faculty have committed to program of peer evaluation for 
professional growth and annual programs of staff development. The Educational 
Leadership Department has developed new procedures for the approval of dissertation 
proposals, has adopted a revised dissertation manual for students, and will provide more 
systematic opportunities for a cohort to meet. The limited resources of the program 
prohibit broadening the admissions criteria and providing more resources to support full- 
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The size of the required seminars will be reduced by offering a second section of 
the seminars. The operational objectives for the 1996-1997 School Year focus on 
strengthening the range of instructional strategies that the faculty will use. The concern 
about gender and racial bias is troubling. Rumors have circulated about these problems 
in the School of Education, but we have not addressed them directly. This is a major 
challenge for the faculty of the School of Education. I hope we can meet it. 

Despite that limitation and the other challenges that we face, ECU’s School of 
Education has made significant progress in the implementation of the Ed. D. in 
Educational Leadership. The findings of the comprehensive evaluation and the 
professional success of our students, e.g., one of the current students in the Ed. D. 
Program in Educational Leadership was selected as North Carolina Principal of the Year, 
support that conclusion. We believe that faculty commitment to disciplined inquiry and 
reflective practice will be central in the program’s continued growth and development. 
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Appendix A 



Ed. D. GRADUATES 


YEAR 


DISSERTATION TOPIC 


Nancy Lucille Daniel 


1996 


Field Testing Science Curriculum 


Karen Robbins Dawkins 


1996 


Fostering Teachers’ Understandings of 
the Nature of Science: A Professional 
Development Model 


Jerry L. Everhart 


1995 


Development and Evaluation of a Staff 
Development Model Addressing Issues of 
Concerns, Beliefs, and Practices, of Middle 
Grades Teachers. 


Mary Whitaker Fox 


1995 


A Model of a Community Program for 
African American Males. 


Mark Raymond Marquez 


1996 


A Competency Based Needs Assessment of 
the Continuing Education for Certified 
Clinical Social Workers. 


Lienne Federico Medford 


1996 


A Curriculum Development Response to the 
North Carolina Public Professional Practices 
Commission’s Call for Cultural Diversity. 


Geraldine Campbell Munn 


1996 


Preparing Teachers for Culturally Diverse 
Classrooms: Multicultural Education for 
Preservice Teachers. 


Joseph Anthony Nelson 


1993 


An Evaluation of Staff Development 
Activities of the Northeast Technical 
Assistance Center. 


Laurie Jean Spring 


1996 


An Assessment of the Perception and 
Awareness of Quality in the Johnson County 
School System. 


Brenda Strickland Tinkham 


1994 


The Impact of Alternative Methods of 



Teacher Evaluation on Teacher Perception 
of Organizational Climate, teacher 
Instructional Practices, and Teacher Self- 
Esteem. 
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Charles R. Watson 



Rodman J. Weston, Jr. 



Susan E. Whitfield 



Roger L. Whitley 



O 

ERIC 



1992 



1993 



1995 



1995 



Attitudes of Middle School Teachers 
Toward Practices Recommended for Middle 
Schools. 

The Trifactor Model of Leadership in 
Secondary Schools: Organizational 
Harmony as a function of Leader Behavior, 
Follower Maturity, and Event Severity. 

A Study of Reasons for Student 
Nonattendance in four rural high schools in 
Eastern North Carolina. 

A Comparison of the Educational 
Performances of Athletes and Nonathletes in 
133 North Carolina High Schools. 
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Appendix B 

STUDENT EVALUATION SURVEY 

ECU SCHOOL OF EDUCATION Ed. D. PROGRAM IN EDUCATIONAL 

LEADERSHIP 



Please circle the year that you were admitted to the Ed. D. Program: 



1990 1991 1992 1993 1994 1995. 

1 . The academic rigor of the program and individual courses is appropriate for the 

development of scholarly practitioners of educational leadership. 

12 3 4 

Strongly Agree Agree Disagree Strongly Disagree 

2. My advising in this program has been informative and helpful 

12 3 4 

Strongly Agree Agree Disagree Strongly Disagree 

3. My class cohort is a cohesive group; we cooperate with one another. 



4. 



5. 



12 3 4 

Strongly Agree Agree Disagree Strongly Disagree 

The faculty treat me with respect and courtesy. 



12 3 4 

Strongly Agree Agree Disagree Strongly Disagree 

I have had adequate access and interaction with the faculty who have taught 
courses in the program. 



12 3 4 

Strongly Agree Agree Disagree Strongly Disagree 

6. I have had adequate access and interaction with other faculty in the Department 
and School. 



1 2 

Strongly Agree Agree 



3 4 

Disagree Strongly Disagree 
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7. I have learned a great deal about the translation of theory into effective practice 
during my internship. 

12 3 4 

Strongly Agree Agree Disagree Strongly Disagree 

8. The scheduling of courses has been logical, well-organized, 
and effective. 

12 3 4 

Strongly Agree Agree Disagree Strongly Disagree 

9. The research methodology courses have been effective. 

12 3 4 

Strongly Agree Agree Disagree Strongly Disagree 

1 0. What recommendations do you have for the improvement of the 
Ed. D. program in educational leadership. 
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Appendix C 

Interview Protocol for the Evaluation 
of the Ed. D. Program in Educational Leadership 



Academic Rigor 

1 . Do you feel that the program has challenged you to perform up to your potential. 
If so, please give me an example. If not please explain. 

2. Are there areas in the program you feel are too easy or not at the level you think 
is appropriate for doctoral level training? Explain. 

3. Do you believe that the courses in research methodology have 

been effective in preparing you to conduct action research in schools and to 
conduct the research for your dissertation? Explain. 

4. If you have knowledge of other doctoral programs, e.g., 

UNC Chapel Hill, how does this program compare in terms of curriculum and 
other academic requirements? 

5. Would you recommend this program to a friend? 

Advising 



1 . Who is your advisor? (Program and/or Dissertation) 

2. How often do you meet with your advisor? Has he or she been 
available? 

3. Has your advisor been helpful in the following areas: 

Selecting courses 

Selecting a dissertation topic 

Reviewing your proposal 

Making referrals when appropriate to other faculty for 

assistance 

Sharing relevant and important information that will 

assist you in managing matriculation issues. 

4. Do you feel that your advisor has some investment in your succeeding in the 
program? 
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Cohesion and Cooperation of the Cohorts 

1 . Have you become acquainted with other students in the program? 

2. Do you work together in study groups or assist one another in obtaining 
assignments and reading materials by sharing notes, etc.? 

3. On a scale of 1 to 5 (with 5 being the highest), how would you rank your class in 
terms of cohesion and cooperation? 

Interaction with Faculty 



1. Do you feel comfortable going to your instructor(s) for assistance or clarification 
of course assignments, material covered in lectures or assigned reading materials, 
etc.? 

2. Do you feel that you can disagree with your instructor(s) without the threat of 
reprisal or being subject o ridicule? 

3. Do you know faculty members other than your instructors? If 
so in what capacity? 

4. Do you feel comfortable approaching other faculty members with whom you have 
not taken a course for assistance? 

5. On a scale of 1 to 5 (with 5 being the highest), how would rank the graduate 
faculty on being approachable by students? 

Internship 



1 . Do you feel that the program offers internships that allow you to continue to 
grow in terms of your professional development? 

2. Is there sufficient diversity in the internship experience to meet the interests 
and needs of the students? 

3. Is the program open to suggestions for other possible internship sites or 
experiences? 

4. Do you feel that you have or should have a voice in deciding upon an internship? 

5. Has the supervision been adequate for your internship? 
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Personal Growth and Development 

1 . Do you feel the faculty is interested in your well-being as a person? 

2. Have you at times felt that your personal needs have been ignored or 
minimized? Explain. 

3. Does the program offer seminars, workshops counseling or support groups that 
deal with personal growth and development? Or is the information about such 
activities made available to you? 

4. Do you feel you have grown or changed in a positive way direction as a result of 
your experience in the program? 

Professional growth and Development 

1 . Does the program provide opportunities or give suggestions for professional 
growth? If so, what is offered? 

2. Who has contributed most to your professional growth and development? 

3. What experience in your doctoral training has contributes most to your 
professional growth and development? 

Residence Requirement 

1. What are your feelings about the residence requirement? 

2. Are there advantages and/or disadvantages to the requirement? 

3. Have the requirements directly affected you? If so, how? 

Scheduling of Courses 

1 . Do you feel the courses are scheduled in an organized manner that contributes to 
the successful completion of the program? 

2. What changes if any would you suggest regarding the scheduling of courses? 
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Socialization in the Culture of the Program 

1 . Describe the social atmosphere in the program? Is it supportive? Is it 
competitive? 

2. Do you feel that certain behaviors, attitudes or requirements, spoken or unspoken, 
that you have to perform or exhibit in order to feel a part of the program or that 
will help you succeed in the program? 

Support and Assistance 

1 . Do you feel the program provides adequate support and assistance for your 
academic needs? Emotional needs? Financial needs? Explain how these needs 
have or have not been met? 

2. If not, what are some recommendations for improvement that will allow the 
program to better meet your needs in the future? 

Recommendations 



What are three recommendations for improving the overall 
doctoral program? Give them in order of priority. 
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